{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing Thinc models (with shape inference)\n",
    "\n",
    "This is a simple notebook showing how you can easily visualize your Thinc models and their inputs and outputs using [Graphviz](https://www.graphviz.org/) and [`pydot`](https://github.com/pydot/pydot). If you're installing `pydot` via the notebook, make sure to restart your kernel (or Google Colab VM – [see here](https://stackoverflow.com/questions/49853303/how-to-install-pydot-graphviz-on-google-colab) for details) after installation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install \"thinc>=8.0.0a0\" pydot graphviz svgwrite"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by defining a model with a number of layers chained together using the `chain` combinator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "from thinc.api import chain, expand_window, Relu, Maxout, Linear, Softmax\n",
    "\n",
    "n_hidden = 32\n",
    "dropout = 0.2\n",
    "\n",
    "model= chain(\n",
    "    expand_window(3),\n",
    "    Relu(nO=n_hidden, dropout=dropout, normalize=True),\n",
    "    Maxout(nO=n_hidden * 4),\n",
    "    Linear(nO=n_hidden * 2),\n",
    "    Relu(nO=n_hidden, dropout=dropout, normalize=True),\n",
    "    Linear(nO=n_hidden),\n",
    "    Relu(nO=n_hidden, dropout=dropout),\n",
    "    Softmax(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the visualization we want to achieve for this model: the **name of the layer** or combination of layers and the **input and output dimensions**. Note that `>>` refers to a chaining of layers.\n",
    "\n",
    "![](https://user-images.githubusercontent.com/13643239/72763790-b8ab3900-3be5-11ea-8ea7-91e4e9fbb181.png)\n",
    "\n",
    "This means we need to add a node for each layer, edges connecting the nodes to the previous node (except for the first/last), and labels like `\"name|(nO,nI)\"` – for instance, `\"maxout|(128,32)\"`. Here's a simple function that takes a Thinc layer (i.e. a `Model` instance) and returns a label with the layer name and its dimensions, if available:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_label(layer):\n",
    "    layer_name = layer.name\n",
    "    nO = layer.get_dim(\"nO\") if layer.has_dim(\"nO\") else \"?\"\n",
    "    nI = layer.get_dim(\"nI\") if layer.has_dim(\"nI\") else \"?\"\n",
    "    return f\"{layer.name}|({nO}, {nI})\".replace(\">\", \"&gt;\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now use `pydot` to create a visualization for a given model. You can customize the direction of the notes by setting `\"rankdir\"` (e.g. `\"TB\"` for \"top to bottom\") and adjust the font and arrow styling. To make the visualization render nicely in a notebook, we can call into IPython's utilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pydot\n",
    "from IPython.display import SVG, display\n",
    "\n",
    "def visualize_model(model):\n",
    "    dot = pydot.Dot()\n",
    "    dot.set(\"rankdir\", \"LR\")\n",
    "    dot.set_node_defaults(shape=\"record\", fontname=\"arial\", fontsize=\"10\")\n",
    "    dot.set_edge_defaults(arrowsize=\"0.7\")\n",
    "    nodes = {}\n",
    "    for i, layer in enumerate(model.layers):\n",
    "        label = get_label(layer)\n",
    "        node = pydot.Node(layer.id, label=label)\n",
    "        dot.add_node(node)\n",
    "        nodes[layer.id] = node\n",
    "        if i == 0:\n",
    "            continue\n",
    "        from_node = nodes[model.layers[i - 1].id]\n",
    "        to_node = nodes[layer.id]\n",
    "        if not dot.get_edge(from_node, to_node):\n",
    "            dot.add_edge(pydot.Edge(from_node, to_node))\n",
    "    display(SVG(dot.create_svg()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling `visualize_model` on the model defined above will render the visualization. However, most dimensions will now show up as `(?, ?)`, instead of the *actual* dimensions as shown in the graph above. That's because Thinc allows **defining models with missing shapes** and is able to **infer the missing shapes from the data** when you call `model.initialize`. The model visualized here doesn't define all its shapes, so the labels are incomplete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"48pt\" viewBox=\"0.00 0.00 910.81 48.00\" width=\"911pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g class=\"graph\" id=\"graph0\" transform=\"scale(1 1) rotate(0) translate(4 44)\">\n",
       "<title>G</title>\n",
       "<polygon fill=\"white\" points=\"-4,4 -4,-44 906.81,-44 906.81,4 -4,4\" stroke=\"transparent\"/>\n",
       "<!-- 96 -->\n",
       "<g class=\"node\" id=\"node1\">\n",
       "<title>96</title>\n",
       "<polygon fill=\"none\" points=\"0,-0.5 0,-39.5 87.72,-39.5 87.72,-0.5 0,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"43.86\" y=\"-27.5\">expand_window</text>\n",
       "<polyline fill=\"none\" points=\"0,-20 87.72,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"43.86\" y=\"-8\">(?, ?)</text>\n",
       "</g>\n",
       "<!-- 101 -->\n",
       "<g class=\"node\" id=\"node2\">\n",
       "<title>101</title>\n",
       "<polygon fill=\"none\" points=\"123.72,-0.5 123.72,-39.5 258.13,-39.5 258.13,-0.5 123.72,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"190.92\" y=\"-27.5\">relu&gt;&gt;layernorm&gt;&gt;dropout</text>\n",
       "<polyline fill=\"none\" points=\"123.72,-20 258.13,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"190.92\" y=\"-8\">(?, ?)</text>\n",
       "</g>\n",
       "<!-- 96&#45;&gt;101 -->\n",
       "<g class=\"edge\" id=\"edge1\">\n",
       "<title>96-&gt;101</title>\n",
       "<path d=\"M87.99,-20C97,-20 106.73,-20 116.48,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"116.58,-22.45 123.58,-20 116.58,-17.55 116.58,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 102 -->\n",
       "<g class=\"node\" id=\"node3\">\n",
       "<title>102</title>\n",
       "<polygon fill=\"none\" points=\"294.13,-0.5 294.13,-39.5 348.13,-39.5 348.13,-0.5 294.13,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"321.13\" y=\"-27.5\">maxout</text>\n",
       "<polyline fill=\"none\" points=\"294.13,-20 348.13,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"321.13\" y=\"-8\">(128, ?)</text>\n",
       "</g>\n",
       "<!-- 101&#45;&gt;102 -->\n",
       "<g class=\"edge\" id=\"edge2\">\n",
       "<title>101-&gt;102</title>\n",
       "<path d=\"M258.28,-20C268.21,-20 278.06,-20 286.89,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"287.05,-22.45 294.05,-20 287.05,-17.55 287.05,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 103 -->\n",
       "<g class=\"node\" id=\"node4\">\n",
       "<title>103</title>\n",
       "<polygon fill=\"none\" points=\"384.13,-0.5 384.13,-39.5 438.13,-39.5 438.13,-0.5 384.13,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"411.13\" y=\"-27.5\">linear</text>\n",
       "<polyline fill=\"none\" points=\"384.13,-20 438.13,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"411.13\" y=\"-8\">(64, ?)</text>\n",
       "</g>\n",
       "<!-- 102&#45;&gt;103 -->\n",
       "<g class=\"edge\" id=\"edge3\">\n",
       "<title>102-&gt;103</title>\n",
       "<path d=\"M348.53,-20C357.49,-20 367.61,-20 377.02,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"377.05,-22.45 384.05,-20 377.05,-17.55 377.05,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 108 -->\n",
       "<g class=\"node\" id=\"node5\">\n",
       "<title>108</title>\n",
       "<polygon fill=\"none\" points=\"474.13,-0.5 474.13,-39.5 608.54,-39.5 608.54,-0.5 474.13,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"541.33\" y=\"-27.5\">relu&gt;&gt;layernorm&gt;&gt;dropout</text>\n",
       "<polyline fill=\"none\" points=\"474.13,-20 608.54,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"541.33\" y=\"-8\">(?, ?)</text>\n",
       "</g>\n",
       "<!-- 103&#45;&gt;108 -->\n",
       "<g class=\"edge\" id=\"edge4\">\n",
       "<title>103-&gt;108</title>\n",
       "<path d=\"M438.38,-20C446.89,-20 456.79,-20 467.01,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"467.12,-22.45 474.12,-20 467.12,-17.55 467.12,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 109 -->\n",
       "<g class=\"node\" id=\"node6\">\n",
       "<title>109</title>\n",
       "<polygon fill=\"none\" points=\"644.54,-0.5 644.54,-39.5 698.54,-39.5 698.54,-0.5 644.54,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"671.54\" y=\"-27.5\">linear</text>\n",
       "<polyline fill=\"none\" points=\"644.54,-20 698.54,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"671.54\" y=\"-8\">(32, ?)</text>\n",
       "</g>\n",
       "<!-- 108&#45;&gt;109 -->\n",
       "<g class=\"edge\" id=\"edge5\">\n",
       "<title>108-&gt;109</title>\n",
       "<path d=\"M608.69,-20C618.61,-20 628.47,-20 637.3,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"637.46,-22.45 644.46,-20 637.46,-17.55 637.46,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 112 -->\n",
       "<g class=\"node\" id=\"node7\">\n",
       "<title>112</title>\n",
       "<polygon fill=\"none\" points=\"734.54,-0.5 734.54,-39.5 812.81,-39.5 812.81,-0.5 734.54,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"773.67\" y=\"-27.5\">relu&gt;&gt;dropout</text>\n",
       "<polyline fill=\"none\" points=\"734.54,-20 812.81,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"773.67\" y=\"-8\">(?, ?)</text>\n",
       "</g>\n",
       "<!-- 109&#45;&gt;112 -->\n",
       "<g class=\"edge\" id=\"edge6\">\n",
       "<title>109-&gt;112</title>\n",
       "<path d=\"M698.58,-20C707.35,-20 717.4,-20 727.22,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"727.3,-22.45 734.3,-20 727.3,-17.55 727.3,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 113 -->\n",
       "<g class=\"node\" id=\"node8\">\n",
       "<title>113</title>\n",
       "<polygon fill=\"none\" points=\"848.81,-0.5 848.81,-39.5 902.81,-39.5 902.81,-0.5 848.81,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"875.81\" y=\"-27.5\">softmax</text>\n",
       "<polyline fill=\"none\" points=\"848.81,-20 902.81,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"875.81\" y=\"-8\">(?, ?)</text>\n",
       "</g>\n",
       "<!-- 112&#45;&gt;113 -->\n",
       "<g class=\"edge\" id=\"edge7\">\n",
       "<title>112-&gt;113</title>\n",
       "<path d=\"M812.81,-20C822.23,-20 832.25,-20 841.43,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"841.57,-22.45 848.57,-20 841.57,-17.55 841.57,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_model(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To fill in the missing shapes, we can call `model.initialize` with examples of the expected input `X` and output `Y`. Running `visualize_model` again now shows the complete shapes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"48pt\" viewBox=\"0.00 0.00 914.85 48.00\" width=\"915pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g class=\"graph\" id=\"graph0\" transform=\"scale(1 1) rotate(0) translate(4 44)\">\n",
       "<title>G</title>\n",
       "<polygon fill=\"white\" points=\"-4,4 -4,-44 910.85,-44 910.85,4 -4,4\" stroke=\"transparent\"/>\n",
       "<!-- 96 -->\n",
       "<g class=\"node\" id=\"node1\">\n",
       "<title>96</title>\n",
       "<polygon fill=\"none\" points=\"0,-0.5 0,-39.5 87.72,-39.5 87.72,-0.5 0,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"43.86\" y=\"-27.5\">expand_window</text>\n",
       "<polyline fill=\"none\" points=\"0,-20 87.72,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"43.86\" y=\"-8\">(?, ?)</text>\n",
       "</g>\n",
       "<!-- 101 -->\n",
       "<g class=\"node\" id=\"node2\">\n",
       "<title>101</title>\n",
       "<polygon fill=\"none\" points=\"123.72,-0.5 123.72,-39.5 258.13,-39.5 258.13,-0.5 123.72,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"190.92\" y=\"-27.5\">relu&gt;&gt;layernorm&gt;&gt;dropout</text>\n",
       "<polyline fill=\"none\" points=\"123.72,-20 258.13,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"190.92\" y=\"-8\">(32, 5488)</text>\n",
       "</g>\n",
       "<!-- 96&#45;&gt;101 -->\n",
       "<g class=\"edge\" id=\"edge1\">\n",
       "<title>96-&gt;101</title>\n",
       "<path d=\"M87.99,-20C97,-20 106.73,-20 116.48,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"116.58,-22.45 123.58,-20 116.58,-17.55 116.58,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 102 -->\n",
       "<g class=\"node\" id=\"node3\">\n",
       "<title>102</title>\n",
       "<polygon fill=\"none\" points=\"294.13,-0.5 294.13,-39.5 350.15,-39.5 350.15,-0.5 294.13,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"322.14\" y=\"-27.5\">maxout</text>\n",
       "<polyline fill=\"none\" points=\"294.13,-20 350.15,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"322.14\" y=\"-8\">(128, 32)</text>\n",
       "</g>\n",
       "<!-- 101&#45;&gt;102 -->\n",
       "<g class=\"edge\" id=\"edge2\">\n",
       "<title>101-&gt;102</title>\n",
       "<path d=\"M258.42,-20C268.28,-20 278.08,-20 286.9,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"287.05,-22.45 294.05,-20 287.05,-17.55 287.05,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 103 -->\n",
       "<g class=\"node\" id=\"node4\">\n",
       "<title>103</title>\n",
       "<polygon fill=\"none\" points=\"386.15,-0.5 386.15,-39.5 442.18,-39.5 442.18,-0.5 386.15,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"414.16\" y=\"-27.5\">linear</text>\n",
       "<polyline fill=\"none\" points=\"386.15,-20 442.18,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"414.16\" y=\"-8\">(64, 128)</text>\n",
       "</g>\n",
       "<!-- 102&#45;&gt;103 -->\n",
       "<g class=\"edge\" id=\"edge3\">\n",
       "<title>102-&gt;103</title>\n",
       "<path d=\"M350.64,-20C359.62,-20 369.71,-20 379.11,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"379.14,-22.45 386.14,-20 379.14,-17.55 379.14,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 108 -->\n",
       "<g class=\"node\" id=\"node5\">\n",
       "<title>108</title>\n",
       "<polygon fill=\"none\" points=\"478.18,-0.5 478.18,-39.5 612.58,-39.5 612.58,-0.5 478.18,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"545.38\" y=\"-27.5\">relu&gt;&gt;layernorm&gt;&gt;dropout</text>\n",
       "<polyline fill=\"none\" points=\"478.18,-20 612.58,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"545.38\" y=\"-8\">(32, 64)</text>\n",
       "</g>\n",
       "<!-- 103&#45;&gt;108 -->\n",
       "<g class=\"edge\" id=\"edge4\">\n",
       "<title>103-&gt;108</title>\n",
       "<path d=\"M442.25,-20C450.79,-20 460.67,-20 470.84,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"470.93,-22.45 477.92,-20 470.92,-17.55 470.93,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 109 -->\n",
       "<g class=\"node\" id=\"node6\">\n",
       "<title>109</title>\n",
       "<polygon fill=\"none\" points=\"648.58,-0.5 648.58,-39.5 702.58,-39.5 702.58,-0.5 648.58,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"675.58\" y=\"-27.5\">linear</text>\n",
       "<polyline fill=\"none\" points=\"648.58,-20 702.58,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"675.58\" y=\"-8\">(32, 32)</text>\n",
       "</g>\n",
       "<!-- 108&#45;&gt;109 -->\n",
       "<g class=\"edge\" id=\"edge5\">\n",
       "<title>108-&gt;109</title>\n",
       "<path d=\"M612.74,-20C622.66,-20 632.52,-20 641.35,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"641.5,-22.45 648.5,-20 641.5,-17.55 641.5,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 112 -->\n",
       "<g class=\"node\" id=\"node7\">\n",
       "<title>112</title>\n",
       "<polygon fill=\"none\" points=\"738.58,-0.5 738.58,-39.5 816.85,-39.5 816.85,-0.5 738.58,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"777.72\" y=\"-27.5\">relu&gt;&gt;dropout</text>\n",
       "<polyline fill=\"none\" points=\"738.58,-20 816.85,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"777.72\" y=\"-8\">(32, 32)</text>\n",
       "</g>\n",
       "<!-- 109&#45;&gt;112 -->\n",
       "<g class=\"edge\" id=\"edge6\">\n",
       "<title>109-&gt;112</title>\n",
       "<path d=\"M702.63,-20C711.4,-20 721.45,-20 731.27,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"731.35,-22.45 738.35,-20 731.35,-17.55 731.35,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "<!-- 113 -->\n",
       "<g class=\"node\" id=\"node8\">\n",
       "<title>113</title>\n",
       "<polygon fill=\"none\" points=\"852.85,-0.5 852.85,-39.5 906.85,-39.5 906.85,-0.5 852.85,-0.5\" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"879.85\" y=\"-27.5\">softmax</text>\n",
       "<polyline fill=\"none\" points=\"852.85,-20 906.85,-20 \" stroke=\"black\"/>\n",
       "<text font-family=\"arial\" font-size=\"10.00\" text-anchor=\"middle\" x=\"879.85\" y=\"-8\">(10, 32)</text>\n",
       "</g>\n",
       "<!-- 112&#45;&gt;113 -->\n",
       "<g class=\"edge\" id=\"edge7\">\n",
       "<title>112-&gt;113</title>\n",
       "<path d=\"M816.86,-20C826.28,-20 836.3,-20 845.47,-20\" fill=\"none\" stroke=\"black\"/>\n",
       "<polygon fill=\"black\" points=\"845.62,-22.45 852.62,-20 845.62,-17.55 845.62,-22.45\" stroke=\"black\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy\n",
    "\n",
    "X = numpy.zeros((5, 784), dtype=\"f\")\n",
    "Y = numpy.zeros((54000, 10), dtype=\"f\")\n",
    "model.initialize(X=X, Y=Y)\n",
    "\n",
    "visualize_model(model)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7.2 64-bit ('.env': venv)",
   "language": "python",
   "name": "python37"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
